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Abstract 
Background 

Cooperation is of utmost importance to society as a whole, but is often challenged by individual self- 
interests. While game theory has studied this problem extensively, there is little work on interactions 
within and across groups with different preferences or beliefs. Yet, people from different social or cultural 
backgrounds often meet and interact. This can yield conflict, since behavior that is considered cooperative 
by one population might be perceived as non-cooperative from the viewpoint of another. 

Methodology and Principal Findings 

To understand the dynamics and outcome of the competitive interactions within and between groups, 
we study game-dynamical replicator equations for multiple populations with incompatible interests and 
different power (be this due to different population sizes, material resources, social capital, or other 
factors). These equations allow us to address various important questions: For example, can cooperation 
in the prisoner's dilemma be promoted, when two interacting groups have different preferences? Under 
what conditions can costly punishment, or other mechanisms, foster the evolution of norms? When does 
cooperation fail, leading to antagonistic behavior, conflict, or even revolutions? And what incentives are 
needed to reach peaceful agreements between groups with conflicting interests? 

Conclusions and Significance 

Our detailed quantitative analysis reveals a large variety of interesting results, which are relevant for 
society, law and economics, and have implications for the evolution of language and culture as well. 



Introduction 

In order to gain a better understanding of factors preventing or promoting cooperation among humans 
or other species, biologists, economists, social scientists, mathematicians and physicists have intensively 
studied game theoretical problems such as the prisoner's dilemma and the snowdrift game (also known 
as chicken or hawk-dove game) [l}[5]. In all these games, a certain fraction of people or even everyone is 
expected to behave uncooperatively (see Fig. [I]). Therefore, a large amount of research has focused on 
how cooperation can be supported by mechanisms such as 



• repeated interactions [lj|6j|8j, 

• reputation [9 - 12 , 



• clusters of cooperative individuals 13 14 , 
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• sanctioning 15-21 , 



• success-driven migration 22 

• economic incentives 



, or 
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For a discussion and classification of cooperation-promoting mechanisms within an evolutionary game- 



theoretical framework see Refs. 3124 25 
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Figure 1. Illustration of the types and outcomes of a game-dynamical treatment of 
one-population symmetrical 2x2 games as a function of the two model parameters B and 



C (see, e.g., 26 , pp. 28ff). B and C are related to the four payoffs P, R, S, and T of symmetrical 2x2 
games via the relations B — S - P and C — R - T, see Sec. Methods for details. The payoff is R, if two 
interacting individuals show behavior 1, but P, if both of them show behavior 2. When the interaction 
partners show different behaviors, the one showing behavior 1 receives the payoff S and the other one 
the payoff T. In the prisoner's dilemmas (PD), behavior 1 corresponds to cooperation and behavior 2 to 
defection. R is then called the "reward" for mutual cooperation, P the "punishment" for mutual 
defection, T the "temptation" for unilateral defection, and S the "sucker's payoff" for unilateral 
cooperation. As the prisoner's dilemma is characterized by the inequalities T > R > P > S, we have B 
< and C < 0, and defection ("free-riding", "cheating") is the dominant strategy. For snowdrift games 
(SD) with T > R > S > P, which are also called chicken or hawk-dove games, we have B > and C < 
0, and uncooperative behavior is tempting. The stable stationary solution corresponds to a coexistence 
of a fraction p = l-S|/(|B| + |C|) of cooperators with a fraction 1 — p of defectors (uncooperative 
individuals). For harmony games (HG) with R > T > S > P, we have B > and C > 0, and 
everybody will eventually cooperate. Finally, for stag hunt games (SH) with R > T > P > S, which are 
also called assurance games, we have B < and C > 0. Here, cooperation is uncertain, as the situation 
is bistable: If the initial fraction po of cooperators is larger than po = \B\/(\B\ + \C\), everybody is 
expected to cooperate in the end, otherwise everybody will eventually behave uncooperatively. 



Many game-theoretical studies of social cooperation are based on models, in which all individuals 
are assumed to have the same properties. In reality, however, individuals are different. To investigate the 
relevance of this for the resulting outcome and dynamics of social interactions, we will consider that people 
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of different gender, status, age, or cultural background may have heterogeneous preferences (e.g. due to 



framing effects, see 27-30 . We will focus here on the case where the preferences are not only gradually 
different, but where we have two interacting populations with mutually incompatible preferences, which 
cannot be satisfied at the same time. For example, men and women appear to have incompatible interests 
many times. Nevertheless, they normally interact among and between each other on a daily basis. It is 
also more and more common that people with different religious beliefs live and work together, while their 
religions request some mutually incompatible behaviors (in terms of the working days and free days, the 
food one may eat or should avoid, the headgear, or appropriate clothing, etc.). A similar situation applies, 
when people with different mother tongues meet or businessmen from countries with different business 
practices make a deal. In this contribution, we are interested in identifying factors, which determine 
whether two such populations go their own way, find a common agreement, or end up in conflict [31] . 
Moreover, we want to understand the relevance of power in the rivalry of populations |32|. 



Our treatment of heterogeneous preferences is based on multi-population games 33-37 . The simpler 



case where individuals of two different populations interact with each other, while they do not interact 
when belonging to the same population, has been nicely summarized by [26| , pp. 182ff. An earlier pub- 
lication on the self-regulation of behavior in animal societies also considers self-interactions within each 
population [38] . In fact, most applications of multi-population models in evolutionary game theory so 
far were oriented at the interaction of biological species and the study of ecosystems |39| - |43] . Compared 
to these publications, our treatment focuses on social systems, and effects of differences in the power of 
interacting populations are considered. The problem of conflicts between social groups has been studied 



before by bimatrix games 44 such as the "battle of sexes" (see, e.g., 35 and by so-called hypergames 
(see, e.g., 45 . However, the related models appear to be less versatile than the one proposed in the follow- 
ing. The main difference to previous approaches is that we study social interactions between and within 
populations, considering that the power of the involved populations may be different. This generates 
interesting kinds of system dynamics, which do not appear when self-interactions (between individuals 
of the same population) are neglected or if all populations are equally strong. For example, we find that 
it may not only depend on the payoffs, but also on the initial condition, whether the individuals of two 
populations with incompatible preferences finally show a commonly shared behavior (see Sec. Evolution 
of normative behavior in the stag hunt game) . 

Note that this paper presents more than "just another model". First of all, our approach fits partic- 
ularly well into widely established modeling concepts. Second, it bridges between two different modeling 
worlds by unifying features of game-theoretical and opinion dynamics models (see Sec. Discussion of 
previous literature on norms). Third, the model is analytically tractable [46| . Fourth, it contains very 
few parameters, while it describes a variety of different system behaviors (although it was not explicitly 
constructed for this). Despite 3 parameters only (which can be further reduced, since only the signs 
and quotient of the parameters B and C matter), the model shows a surprisingly rich behavior and can 
reproduce a variety of phenomena observed in social systems: (1) the breakdown of cooperation, (2) the 
coexistence of different behaviors (the establishment of "subcultures"), (3) the evolution of commonly 
shared behaviors ("social norms"), and (4) the occurrence of social polarization, conflict, or revolutions. 
The approach can also be cast into an agent-based model. In fact, agent-based models for the establish- 



ment of norms have found a lot of interest, recently 47-60 



Modeling Approach 

The crucial point of our modeling approach is to adapt the game-dynamical replicator equations for mul- 
tiple populations 35, 38, 46, 61 1 in a way that reflects interactions between individuals with incompatible 
preferences (see Sec. Methods). The resulting equations ^ and Q describe the time evolution of the 
proportions p(t) and q(t) of cooperative individuals in populations 1 and 2, respectively, as individuals 
imitate more successful behaviors in their own population. Their success depends on the "payoffs" quan- 
tifying the results of social interactions, i.e., on the own behavior and the behavior of the interaction 
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partner(s). 

In order to reflect incompatible interests of both populations, we assume that population I prefers 
behavior 1, while population 2 prefers behavior 2. If an interaction partner shows the behavior preferred 
by oneself, we call this behavior "cooperative" , otherwise uncooperative. Accordingly, behavior 1 is coop- 
erative from the viewpoint of population 1, but uncooperative from the viewpoint of population 2 (and 
vice versa). Furthermore, if an individual of population 1 interacts with an individual of population 2 
and both display the same behavior, we call this behavior "coordinated" . Finally, if the great majority of 
individuals in both populations shows a coordinated behavior (in case of a commonly shared behavior), we 
speak of "normative behavior" or a "behavioral norm" . To establish a social norm, one of the populations 
has to give up its preferred behavior. 

What will be the resulting dynamics and outcome of such interactions? Under what conditions will we 
find "normative behavior" (although neither the relative sizes of both populations nor their incompatible 
preferences change in our model)? To answer these questions, the payoffs from social interactions in 
2x2 games are represented by T, R, P, and S, as usual (see Sec. Methods for details). In the prisoner's 
dilemma, the meaning of these parameters is "Temptation" to behave non-cooperatively, "Reward" for 
mutual cooperation, "Punishment" for mutual non-cooperative behavior and "Sucker's payoff" for a 
cooperative individual meeting an uncooperative one (see Fig.[l]). The related game-dynamical replicator 
equations contain two payoff-dependent parameters, B — S - P and C = R - T. C may be interpreted 
as gain by coordinating on one's own preferred behavior (if greater than zero, otherwise as loss). B 
reflects the gain when giving up coordinated, but non-preferred behavior. Equations Q and ([6| contain 
a further parameter /, which can be interpreted as "(relative) power" of population 1 (while 1-/ would 
correspond to the relative power of population 2). The relative power may represent the relative size of the 
populations, but also differences in their material resources (money, weapons, etc.), social capital (status, 
social influence, etc.), and other factors (charisma, moral persuasion, etc.). It reflects how much influence 
a population has on the behavioral choice of individuals. When a population has a greater relative power 
than another one, we call it "stronger" , if it has less power, we call it "weaker" . Details of the model and 
some generalizations are provided in Methods. 



Results 

We have solved Eqs. ^ and Q by numerical simulation for different parameter values B, C, and / and 
different initial conditions p(0) and q(0). In contrast to the computer-based analysis presented here, a 
mathematical analysis of the stationary (i.e. time-invariant) solutions and their stability properties has 
been carried out in a complementary paper |46| . While the linear stability analysis reveals the sensitivity 
to stochastic fluctuations (random effects, "noise"), the sensitivity to parameter variations is captured by 
so-called phase diagrams. Here, we will not go into these technicalities, but rather discuss representative 
examples of the different kinds of system dynamics and their relevance for social systems. 

We find that social interactions with incompatible interests do not necessarily produce conflict — 
they may even promote mutual coordination. Depending on the signs of B and C, which determine the 
character of the game, we have four archetypical situations: 

1. In games like the multi-population prisoner's dilemma (MPD), we have B < and C < 0. 

2. In the multi-population harmony game (MHG), we have B > and C > 0. 

3. B < and C > applies to games like the multi-population stag hunt game (MSH). 

4. The multi-population snowdrift game (MSD) is characterized by B > and C < 0. 

In a multi-population prisoner's dilemma with incompatible preferences (MPD), everybody behaves 
non-cooperatively in the end (see Fig.[2|. This does not even change, if one population is stronger than the 
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other one (i.e. / ^ 1/2, or if the interaction rate between populations is different from the interaction rate 
within populations. This disappointing outcome results despite the fact that non-cooperative behavior 
in one population corresponds to cooperative one from the perspective of the other. However, as non- 
cooperative individuals earn a high payoff (the temptation T), when meeting a non-cooperative individual 
of the other population (it is cooperative from the own perspective), there is no incentive to give up 
defection in their own population — on the contrary. 

In contrast, in the multi-population harmony game, everybody finally shows a cooperative behavior 
in the own population, but due to the different preferences, the behaviors in both populations are not 
coordinated. (Every population just does what it likes, as if both populations had their own "subcultures" , 
see Fig. [2]). 



Prisoner's Dilemmas Harmony Games 
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P(t) 
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Figure 2. Simulation results for two interacting populations with self-interactions and 
incompatible preferences playing prisoner's dilemmas or harmony games. The outcome is 
visualized by vector fields (small arrows) and sample trajectories (large arrows) for / = 0.8 (i.e. 
population 1 is assumed to be stronger than population 2). p denotes the fraction of individuals in 
population 1 showing their preferred, cooperative behavior 1, and q is the fraction of cooperative 
individuals in population 2 showing their preferred behavior 2. A fraction 1 - q of individuals in 
population 2 shows the non-preferred behavior 1, and a fraction 1 - p of individuals in population 1 
shows behavior 2. The vector fields show (dp/dt,dg/dt), i.e. the direction and size of the expected 
temporal change of the behavioral distribution, if the fractions of cooperative individuals in populations 
1 and 2 are p{t) and q{t). Sample trajectories illustrate some representative flow lines (p(t),q(t)) as 
time t passes. The flow lines move away from unstable stationary points (empty circles) and are 
attracted towards stable stationary points (black circles). Saddle points (crosses) are attractive in one 
direction, but repulsive in another. The colored areas represent the "basins of attraction", i.e. all initial 
conditions (p(0),q(0)) leading to the same stable fix point [red = (0,0), yellow = (1,1)]. Intuitively, the 
initial conditions quantify the influence of the previous history. (A) If B = C = — 1, the individuals in 
each population are facing prisoner's dilemma interactions and end up with non-cooperative behavior. 
(B) If B = C = 1, individuals in each population are playing a harmony game instead, and everybody 
eventually behaves cooperatively. The results look similar when the same two-population games are 
played with different values of/, |5|or \ C\. 

As we will show in the following, the dynamics and outcome of the multi- population stag hunt 
and snowdrift games with incompatible preferences are more complicated and in marked contrast to 
the corresponding one-population games. This can be demonstrated by systematically exploring the 



() 



parameter space with computer simulations. In the following, we will illustrate typical simulation results 
by figures and movies showing the stationary solutions (fix points, evolutionary equilibria) of the games, 
their basins of attraction, and representative flow lines. Details are discussed below and in the captions 
of Figs. [2}]5] (see also Movies S1-S3 and Methods). 



Evolution of normative behavior in the stag hunt game 



The one-population stag hunt game is characterized by an equilibrium selection problem 62 : Everyone is 
finally expected to cooperate, if the initial fraction of cooperative individuals is above po = \B\/(\B\ + \C\), 
otherwise nobody will behave cooperatively in the end (see Fig. [T]). The same applies to non-interacting 
populations (see Movie SI). For interacting populations without self-interactions, however, it never hap- 
pens in the multi-population stag-hunt game with incompatible preferences that everybody or nobody 
cooperates in both populations (otherwise there should be yellow or red areas in the second part of Movie 
S2). Although both populations prefer different behaviors, all individuals end up coordinating themselves 
on a commonly shared behavior (corresponding to the blue and green areas in Movie S2). This can be 
interpreted as self-organized evolution of a social norm (see below). 

Note that the previously discussed cases, which neglect interactions between populations or within 
populations, are applicable to particular social systems only. Normally, however, there are interactions 
between different populations and, at the same time, interactions between individuals of the same popu- 
lation ( "self- interactions" ) . If this is taken into account, the case where everybody or nobody cooperates 
in both populations is still possible, but it requires that both populations have similar power (/ sa 1/2) 
and that the initial levels of cooperation, p(0) and g(0), are comparable as well. Under such conditions, 
both populations may develop separate, coexisting norms (see yellow area in Fig. [3j3 and Movie S3), 
as for the multi-population harmony game. Normally, however, both populations establish a commonly 
shared norm and either end up with behavior 1 (green area in Fig. [3]) or with behavior 2 (blue area). 

The behavior of Eqs. j5]) and ([6| becomes better understandable for multi-population games with the 
payoffs P = c, R = b+c, S = b, and T = 0, where the payoff b reflects the benefit of showing the preferred 
behavior, while c is the payoff for showing coordinated behavior (reflecting the reward for conforming 
with the behavior of the interaction partner). While these payoffs do not correspond to a stag hunt game, 
for c > and -c < b < c they also lead to B = b-c < and C = b+c > 0, which implies exactly the 
same solutions. One advantage of this formulation besides the better interpretation is the possibility to 
extend it to simultaneous interactions with several players. 

For the sake of illustration of this specification, assume that individuals of population 1 like to be 
properly dressed at the beach and individuals of population 2 enjoy to be naked (6>0), but even more than 
doing what they like, everybody prefers to conform with the behavior of the interaction partners (c>b). 
In situations, when naked people and those wearing a swimming suit interact with each other at the same 
part of the beach, our equations allow one to identify conditions under which one population eventually 
sets the behavioral standards or under which a mixture of both behaviors persists. (In contrast to the 
related multi-population game without self-interactions, see Movie S2, there is a possibility that the two 
populations do not coordinate their behaviors, and everybody ends up doing what he or she likes.) Note 
that, if both populations interact in space, dressed people and nudists may segregate, even when there 
was no disapproval between both behaviors. As a consequence, there may be different "(sub-) cultures" in 
different parts of the beach, as is often observed. This becomes understandable by the circumstance that 
the effect of mutual disapproval of the non-preferred behavior (which may be described by the payoffs P 
= 0, R — b, S — b~c, and T = -c) again leads to B = b c < and C = b+c > 0. Therefore, the same 
kind of dynamics results as in the case where there is a tendency to conform with others. 

In conclusion, due to the payoff structure of the multi-population stag hunt game and other multi- 
population games with B < and C > 0, it can be profitable to coordinate oneself with the prevailing 
behavior in the other population. Yet, the establishment of a norm requires the individuals of one pop- 
ulation to give up their own preferred behavior in favor of the one preferred by the other population. 
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Stag Hunt Games 




P(t) p(t) 

Figure 3. Simulation results for two interacting populations with self-interactions and 
incompatible preferences, playing stag hunt games. The corresponding vector fields (small 
arrows), sample trajectories (large arrows) and phase diagrams (colored areas) were determined for B < 
and C > 0. The representation is the same as in Fig. [2j In particular, the colored areas represent the 
basins of attraction, i.e. all initial conditions (p(0),q(0)) leading to the same stable fix point (stationary 
solutions) [yellow = (1,1), blue = (0,1), green = (1,0)]. The dashed diagonal line represents an infinite 
number of unstable fix points. The model parameters are as follows: (A) \B\= \ C\= 1 and / = 0.8, i.e. 
population 1 is more powerful than population 2, (B) \ C\= 2\B\= 2 and / = 1/2, i.e. both populations 
are equally strong, (C) \C\ = 2\B\= 2 and / = 0.8, (D) 2\C\= \B\= 2 and / = 0.8. Due to the 
asymptotically stable fix points at (1,0) and (0,1), all individuals of both populations finally show the 
behavior preferred in population 1, when starting in the green area, or the behavior preferred in 
population 2, when starting in the blue area. This case can be considered to describe the evolution of a 
shared behavioral norm. Only for similarly strong populations (/ s» 1/2) and similar initial fractions 
p(0) and q(0) of cooperators in both populations (yellow area), both populations will end up with 
population-specific norms ("subcultures"), corresponding to the asymptotically stable point at (1,1). 
The route towards the establishment of a shared norm may be quite unexpected, as the flow line 
starting with the white circle shows: The fraction q(t) of individuals in population 2 who are 
uncooperative from the viewpoint of population 1 may grow in the beginning, but later on go down 
dramatically. Therefore, a momentary trend does not allow one to easily predict the final outcome of 
the struggle between two interest groups. 

Therefore, it is striking that the preferred behavior of the weaker population can actually win through 
and finally establish the norm (see blue areas in Figs. [3]^,C,D). Who adapts to the preferred strategy 
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of the other population essentially depends on the initial fractions of behaviors (and, thereby, on the 
previous history). The majority behavior in the beginning is likely to determine the resulting behavioral 
norm, but a powerful population is in a favorable position: The area of possible histories leading to an 
establishment of the norm preferred by population 1 tends to increase with power / (compare the size of 
the green areas in Figs. [3^3+C). 



Discussion of the equilibrium selection problem 

As was indicated already, when two populations with incompatible preferences interact among and be- 
tween each other, the behavior of the stag hunt game changes completely: Then, the values of the payoff- 
dependent parameters B and C have an influence on the stable stationary solutions, and inner stationary 
points [p, q) with < p, q < 1 disappear, if \B\ ^ |C| [46] . 

It is noteworthy that, without interactions between populations, the stag hunt game implies an inter- 
esting equilibrium selection problem [6| |100 - 102] , since it has several stable solutions. These are classified 



as payoff-dominant solution (which maximizes the individual payoff, if the interaction partner decides in 
the same way) and risk-dominant solution (which "minimizes the maximum damage" , i.e. maximizes the 
individual payoff for the worst-case choice of the interaction partner and corresponds to non-cooperative 
behavior). Which of these solutions is selected depends on the initial conditions, and there is a monotonous 
increase or decrease of the fraction p(t) of cooperative individuals in the course of time t. In the one- 
population stag hunt game, the payoff-dominant solution corresponds to cooperative behavior by every- 
body (lim^oo p(t) = 1). It is selected, if the initial fraction p(0) of cooperative individuals is above the 
value po = |i?|/(|B|-(-|C|) of the unstable stationary solution. Instead, the risk-dominant solution results 
for p(0) < po and corresponds to non-cooperative behavior by everyone (lim t _ i . 00 p(t) = 0) . 

In the multi-population games with interactions across populations, the risk-dominance concept is 
not sufficient to understand the dynamics and outcome of the game. For example, when two populations 
with incompatible preferences play stag hunt games without self-interactions, the game is of the "battle 
of sexes" type, and there are no thresholds that would separate payoff-dominant from risk-dominant 



solutions 35 46 . When both, interactions and self-interactions are considered, the inner stationary point 
disappears whenever \B\ ^ \C\. The unstable solution is rather located at the boundary or in one of the 
corners. Moreover, as Figs. [3p and [3j3 illustrate, the incompatibility of preferences can destabilize the 
solutions (0,0) or (1,1). Even an initial increase in the fractions p(t) and q(t) of cooperative individuals 
(i.e. p'(0) > and q'(0) > 0, where p'(t) = dp(t)/dt and q'(t) — dq(t)/dt) does not imply that the system 
will end up in the stationary solution (1,1). While there still exists a payoff- and a risk-dominant solution 
for the stronger population, there is no threshold behavior for the weaker population, as Figs. [3|3+C 
show. 

In the limiting case where the relative size / of population 1 goes to one, the resulting fraction of 
cooperative individuals in population 2 is completely determined by the initial fraction p(0) of cooperative 
individuals in population 1, while the initial fraction g(0) of cooperative individuals in population 2 does 
not have any influence. No matter whether population 1 selects the payoff-dominant solution (for large 
enough values of p(0)) or the risk-dominant solution (for small values of p(0)), the behavior in population 
2 is always coordinated with population 1 in the end [46] . The risk-dominant case can be interpreted 
such that population 2 effectively manages to set the norm. Figure [4] shows the parameter dependence 
for the general case, using the alternative parametrization B — b — c and C = b + c, where b is the 
benefit of showing the preferred behavior and c is the benefit of showing coordinated behavior. The 
classical coordination game, where individuals always form a behavioral convention (i.e. coordinate on a 



behavioral norm) results for 6 = 61 
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Figure 4. Illustration of the finally resulting system state for the two-population stag hunt 
game with interactions and self-interactions, and incompatible preferences. The outcome is 
displayed as a function of the relative strength / of population 1 and the ratio b/c between the benefit b 
of showing the preferred behavior and the benefit c of showing coordinated behavior. If b > c, 
individuals always show the behavior they prefer. However, if b < c, a commonly shared behavior (i.e. a 
"social norm") may result, depending on the respective initial conditions. The initial commitments p(0) 
and q(0) to the respectively preferred behavior determine also, who sets the norm. If all individuals 
initially show the behavior they prefer (i.e. p(0) = q(0) = 1, as assumed in the above illustration), the 
stronger population sets the norm, given the difference between both population strengths, / and 
(1 — /), is larger than a certain threshold. Otherwise, individuals in each population show their preferred 
behavior. The threshold depends on the relative strength / and the size of b/c, i.e. the fraction of the 
relevant payoffs b and c. It can, in principle, be determined from analytical results derived in Ref. 46 . 



Discussion of previous literature on norms 



Note that the subject of social norms is a multi-faceted research field, and there is no single, generally 
accepted definition of what norms are 



63-66 . Definitions range from the concept of "oughtness" 67 



to a 

behavioral regularity or shared behavior with a sanctioning ("punishment") of non-conforming behavior 
20 68-77 . However, not all authors agree on the necessity of the sanctioning element [78j[79]. In our 
manuscript, we define "normative behavior" or a "behavioral norm" as a situation in which behaviors 
are shared among a large majority of individuals. 

The question of how behavioral consensus may evolve has been addressed by opinion dynamics models 
such as voter models 55 or models of in- and out-group interactions 59|80 . The currently most common 



approach to behavioral norms is based on game theory and relates the issue to ultimatum games 62|65|78 
stag hunt games (8T 



see 



82 , prisoner's dilemmas [15 47 69,83,84], or related concepts 



For overviews 



63 83|85 . Yet, the majority of these models investigate conditions under which people comply with a 



preset norm. Repeated interactions [T] and the sanctioning of non-conforming behavior |86| are two such 
conditions. It was less clear, however, whether and how a commonly shared norm would be established 
in situations, where the involved populations prefer different norms. 
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Our own approach to understanding behavioral consensus relates to evolutionary game theory, but in 
contrast to most models for the evolution of norms, it assumes a heterogeneity of individual preferences 
and therefore involves several populations. In Methods and Discussion, we furthermore consider the 
role of sanctioning as a mechanism that supports the evolution of norms, but point out that there are 
other mechanisms which are expected to support the evolution of norms as well. Note that the term 
"evolution" is used by us in the sense of "temporal evolution" or "eventual establishment" , not in the 
sense of "biological evolution" or "spontaneous emergence" . 



Examples and classification of norms 

In the following, we will illustrate the concept of social norms by some examples. In his book "The 



Cement of Society" , Jon Elster 79 discusses consumption norms, norms against behavior 'contrary to 
nature', norms regulating the use of money, norms of reciprocity, medical ethics, codes of honor, norms 
of retribution, work norms, norms of cooperation, and norms of distribution. Norms underlying common 



neighborhood or business practices have been analyzed by Macaulay [ST , Ostrom 1881 and Ellickson 69 



When discussing norms, it is useful to distinguish between coordination norms and cooperation norms 
83 . Coordination norms are self-enforcing. They are established, when it is advantageous for people to 
show a coordinated behavior, but it does not matter which of the behavioral options people agree upon. 
In that case, one also speaks of behavioral conventions, and one-population models arc often sufficient 



to describe the underlying dynamics 61 89 . Examples of conventions are the preference of pedestrians 
to walk on one side 61 90 -93] (for example, the right-hand side in continental Europe or the left-hand 
side in Japan), the direction of writing, the way people greet each other (whether one gives a hand and 
which one, whether one hugs or kisses the person and how many times), the way people eat, the color 
of clothes worn by political movements, and signs used by followers of certain ideas or tastes to identify 
each other (e.g. tattoos or hanky codes). 

In contrast to coordination norms, cooperation norms are not self-enforcing, since there are incentives 
for unilateral deviance. In our paper, we analyze situations, where people have different preferences, so 
that at least a certain fraction of people is tempted to show non-conforming behavior. Gender norms may 
serve for illustration. Just imagine a "battle of the sexes" in a group of friends (rather than between two 
players), where men prefer to watch soccer and women prefer to see a cultural performance, to discuss 
a stereotypical example. Note that, in our model, interactions occur not only between men and women, 
but also among men and among women, so the outcome will depend on their relative power. 

Religious norms constitute another case, where people with incompatible preferences interact with 
each other. A similar thing applies to legal norms, when people believing in a pluralistic civil law system 
interact with people believing in a religious law system. It is well-known that these law systems have 
incompatible implications with regard to certain issues. A similar situation applies, when businessmen 
from countries with different business practices make a deal or people with different mother languages 
meet. In our opinion, communicating in a language is not just a coordination problem. Most people have a 
clear preference for their mother tongue, and it shapes even the way of thinking and of social interactions. 
Therefore, when people with different mother tongues meet, there is an incentive to unilaterally deviate 
from speaking the same language (e.g., due to differences in proficiency). Nevertheless, a common ("nor- 
mative") language can establish, as is impressively shown, for example, by the unification of regionally 
spoken dialects in Germany triggered by the Luther bible. Note, however, that proper language use does 
not seem to be fully self-enforcing, otherwise lexica, schools, and related legal regulations would not be 
needed. 

Besides coordination and cooperation norms, it appears to make sense to distinguish a third class 
of "hybrid norms", which share features of both kinds of norms. This case occurs when it is costly 
to switch the behavior (i.e. when transaction costs are high). Technological norms may serve as an 
example. Customers will usually profit from shared technical standards concerning, for example, the type 
of keyboard (QWERTY or Dvorak) [94], the kind of operating system (Windows vs. Mac OS or Linux), 
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the technology of video players (VHS vs. Beta MAX) [95] or high resolution DVD players (blue-ray vs. 
HD DVD). In such cases, customers do not have incentives to deviate from a technological standard, once 
it has established everywhere. In the beginning, however, a common standard does not evolve by itself, 
as customers buy different technologies and are reluctant to give up the technology they have invested in. 
Therefore, the use of a single technology is not self-enforcing in the beginning. Once there is a majority 
standard, however, most people will join it after some time, and their preferences change accordingly. 

One should also mention that some conventions or norms are set by law, e.g. the driving side 89 , or 
may involve an intentional segregation from other groups (e.g. when groups develop their own dress-codes 
or symbols such as tattoos). This touches issues of group dynamics, which are beyond the scope of this 
paper. The novel contribution of our model is that it sheds new light on the problem of whether a norm can 
establish (under what conditions) and how (in terms of the dynamics) . There are even exact mathematical 
results for this [46] . In particular, our model reveals that the dynamics and finally resulting state of the 
system is not only determined by the payoff structure. It also depends on the power of populations and 
even on the initial proportions of cooperative individuals (the initial conditions or previous history) . 

Within our model of the evolution of norms, one could say that Figs. [3}\,C,D represent the formation 
of coordination norms, as one behavioral norm is always established (reflecting self-enforcement). Figure 
[3j3, in contrast, describes situations where two different behaviors can coexist in a stable way (see the 
yellow basin of attraction). Due to this lack of self-enforcement, it makes sense to attribute this case to the 
problem of establishing a cooperation norm. This relevant case can only occur, when taking into account 
self-interactions in multi-population games. It is also interesting to note, that the application of group 
pressure can transform the problem of establishing a cooperation norm into the problem of establishing 
a coordination norm (see Sec. Methods). Finally, the case of hybrid norms can be treated by considering 
transaction costs in our model. 



Occurence of social polarization in the snowdrift game 

Let us now turn our attention to the discussion of snowdrift games. In the one-population case, there is 
one stable stationary point, corresponding to a fraction po = |5|/(|S| + |C|) of cooperative individuals 
(see Fig. [I]). If this would be transferable to the multi-population case we are interested in, we should 
have p = q = po in the limit of large times t — > 00. Instead, we find a variety of different outcomes, 
depending on the values of the model parameters B, C, and / (see Fig. [5]) : 

(a) The interactions between both populations shift the fraction of cooperative individuals in each pop- 
ulation to values different from p Q . If \B\= |C|, we discover a line of infinitely many stationary 
points, and the actually resulting stationary solution uniquely depends on the initial condition (see 
Fig.[&|4). This line satisfies the relation q = p only if / = 1/2, while for most parameter combina- 
tions we have g / p ^ po. Nevertheless, the typical outcome in the case \B\= \ C\ is characterized 
by a finite fraction of cooperative individuals in each population. 

(b) Conflicting interactions between two equally strong groups destabilize the stationary solution q = p 
= po of the one- population case, and both populations lose control over the final outcome. For \B\=/= 
\C\, all stationary points are discrete and located on the boundaries, and only one of these points 
is an evolutionary equilibrium. If both populations have equal power (/ = 1/2), we either end up 
with non-cooperative behavior by everybody (if po < 1/2, see Fig. [5j3) , or everybody is cooperative 
(if po > 1/2). Remarkably, there is no mixed stable solution between these two extremes. 

(c) The stronger population gains control over the weaker one, but shows polarization itself, and a 
change of the model parameters may induce a revolution. If \B\^ \C\ and population 1 is much 
stronger than population 2 (i.e., / — 1/2 3> 0), we find a finite fraction of cooperative individuals in 
the stronger population, while either 0% or 100% of the individuals are cooperative in the weaker 
population. A closer analysis reveals that the resulting overall fraction of cooperative individuals 
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Snowdrift Games 




P(t) P(t) 

Figure 5. Two interacting populations with self-interactions and incompatible preferences, 
playing snowdrift games. The corresponding vector fields (small arrows), sample trajectories (large 
arrows) and phase diagrams (colored areas) were determined for B > and C < 0. The flow lines move 
away from unstable stationary points (empty circles) and are attracted towards stable stationary points 
(black circles and solid diagonal line). Saddle points (crosses) are attractive in one direction, but 
repulsive in another. The representation is the same as in Fig. [2] In particular, the colored areas 
represent the basins of attraction, i.e. all initial conditions (p(0),q(0)) leading to the same stable fix 
point [red = (0,0), salmon = (m,0), mustard = (w,l), rainbow colors = (m,w), with < U, v < 1]. The 
model parameters are as follows: (A) |-B|= \ C\= 1 and / = 0.8, i.e. population 1 is more powerful than 
population 2, (B) |C|= 2|5| = 2 and / = 1/2, i.e. both populations are equally strong, (C) |C|= 2|5| = 
2 and / = 0.8, (D) 2\C\= \B\= 2 and / = 0.8. (A) In the multi-population snowdrift game (MSD), a 
mixture of cooperative and uncooperative behaviors results in both populations, if |S|=|C|. (B) For 
|5|<|C|and equally strong populations, everybody ends up with non-cooperative behavior in each 
population. (C) For |i?|< |C|and / — 1/2 S> 0, the weaker population 2 forms a "tacit alliance" with the 
minority of the stronger population 1 and opposes its majority. (D) Same as (C), but now, all 
individuals in the weaker population 2 show their own preferred behavior after the occurrence of a 
"revolutionary" transition, during which the stable stationary solution (the evolutionary equilibrium) 
changes discontinuously from (u,0) to (v,l). 

fits exactly the expectation po of the stronger population [46] , while from the perspective of the 
weaker population, the overall fraction of cooperative individuals is largely different from p — 
|5|/(|S| + |C|). Note that the stronger population alone can not reach an overall level of cooperation 
of pq. The desired outcome can only be produced by effectively controlling the behavior of the 
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weaker population. This takes place in an unexpected way, namely by polarization: The stronger 
population splits up into fractions of people showing different behaviors, which may give rise to 
social differentiation, inequality, and conflict. In the weaker population 2, everyone either shows 
behavior 1 (namely for po < 1/2, see Fig. [5p), otherwise everyone shows behavior 2 (see Fig. [5p). 
There is no solution in between these two extremes (apart from the special case po = 1/2 for \B\ — 

\c\). 

It comes as a further surprise that the behavior in the weaker population is always coordinated with 
the minority behavior in the stronger population. Due to the payoff structure of the multi- population 
snowdrift game, it is profitable for the weaker population to oppose the majority of the stronger pop- 
ulation, which creates a tacit alliance with its minority. Such antagonistic behavior is well-known from 



protest movements 96 and finds here a natural explanation. 

Moreover, when \C\ changes from values greater than \B\ to values smaller than \B\, there is an 
unexpected, discontinuous transition in the weaker population 2 from a state in which everybody is 
cooperative from the point of view of population 1 to a state in which everybody shows the own preferred 



behavior 2 (see Movie S3). History and science 97 have seen many abrupt regime shifts of this kind 



Revolutions caused by class conflict provide ample empirical evidence for their existence. Combining 



the theory of phase transitions with "catastrophe theory" 98 offers a quantitative scientific approach 
to interpret such revolutions as the outcome of social interactions 99 . Here, their recurrence becomes 
understandable in a unified and simple game-theoretical framework. 



Discussion 

Multi-population game-dynamical replicator equations provide an elegant and powerful approach to study 
the dynamics and outcomes expected for populations with incompatible interests. A detailed analysis 
reveals how combining interactions within and between populations and considering differences in their 
power can substantially change the dynamics of various game theoretical dilemmas (compare Movies S2 
and S3 with Movie SI). Generalizations to more than 2 behaviors or groups and to different payoffs for 
in- and out-group interactions are easily possible (see Sec. Methods). 

When two populations with incompatible preferences interact among and between each other, we find 
the same stationary points for the prisoner's dilemma and the harmony game as for the corresponding 
non-interactive games. In particular, interactions across populations do not change the attractive solutions 
(Nash equilibria) of these games. However, the behavior of the snowdrift game and the stag hunt game 
changes completely, and their dynamics is particularly interesting. For the interactive case, the signs of 
the payoff-dependent parameters B and C do not only determine the character of the game, but also the 
location and stability of the stationary solutions, and the basins of attraction. 

In the multi-population snowdrift game, for example, there is a discontinuous ("revolutionary") tran- 
sition, when 1-|5|/| C|changes its sign. On top of this, the power / has a major influence on the outcome, 
and the initial distribution of behaviors (and, consequently, the previous history) can be crucial, also for 
the multi-population stag hunt game. Note that such a rich system behavior is already found for the 
simplest setting of our model and that the concept of multi-population game-dynamical equations may 
be generalized in various ways to address a number of challenging questions in the future: How can we 
gain a better understanding of a clash of cultures, the outbreak of civil wars, or conflicts with ethnic or 
religious minorities? How can we analytically study migration and group competition? When do social 
systems become unstable and experience a polarization of society? How can we understand the emergence 
of fairness norms in bargaining situations? 

Another interesting aspect of our model is that it makes a variety of quantitative predictions. There- 
fore, it could be tested experimentally with iterated games in the laboratory, involving several groups of 
people with random matching and sufficiently many iterations. Suitable changes in the payoff matrices 
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should be able to confirm the mathematical conditions under which different archetypical types of social 
phenomena or discontinuous transitions in the system behavior can occur: (1) the breakdown of cooper- 
ation, (2) in-group cooperation (the formation of "sub-cultures"), (3) the evolution of shared behavioral 
norms, and (4) societal polarization or conflict with the possibility of a revolutionary regime shift. The 
findings are particularly important to understand interactions between human populations with different 
ethnic, cultural or religious backgrounds. However, they are also relevant for social features within animal 
societies 38 103 ■ 105 or even for interactions among bacteria [4l[5] . 



The significant influence of the respective payoffs of social interactions on the resulting outcome has 

There, conflicts need to be avoided or 



crucial implications for society, law and economics 62 106 -115 



solved, and norms and standards are of central importance. For society, norms are equally important 
as cooperation, since they reduce uncertainty, bargaining efforts, and (potentially) also conflict in social 
interactions. They are like social forces guiding our interactions in numerous situations and subtle ways, 
creating an "invisible hand" kind of self-organization of society [47]. Nevertheless, their ubiquity is quite 
surprising, as norms require people to constrain self-interested behavior 116 and to perform socially 
prescribed roles. Yet, widespread cooperation-enhancing mechanisms such as direct reciprocity due to 
repeated interactions [l] and indirect reciprocity based on reputation [10] can transform a prisoner's 
dilemma into stag hunt interactions, see Fig. [6] 1 3 , 82 . 117]. 



Stag Hunt 
(Bistability) @ 



Prisoner's Dilemma 
(Defection) 



Harmony 
(Cooperation) 




^ Snowdrift 
3% (Coexistence) 



Figure 6. Illustration of different routes to cooperation (through arrows), assuming 
particular reproduction-selection mechanisms |3|. The direction of the arrows can be 



mathematically calculated 117 . Route 1 reflects the way in which the payoff-dependent parameters B 
and C of the game (see Fig. [I]) are effectively modified by kin selection (genetic relationship), network 
reciprocity (clustering of individuals showing the same behavior) , or group selection (competition 
between different groups) . Route 2a corresponds to the effect of direct reciprocity (due to the "shadow 
of the future" through the likelihood of future interactions). Route 2b belongs to the mechanism of 
indirect reciprocity (based on reputation effects), and route 2c reflects costly punishment. Route 3 



results for certain kinds of network interactions 



118 



This suggests a natural tendency towards the formation of norms, whatever their content may be. 
Costly punishment can support the evolution of norms in prisoner's dilemma situations as well (see Fig. 
[7]A-). Another way of promoting the preferred coordinated behavior as commonly accepted norm is to 



15 



transform a prisoner's dilemma situation into a stag hunt game in the own population and to make sure 
that the population interacts with another one with incompatible preferences and prisoner's dilemma 
interactions (see Fig. [7^3) . Accordingly, the sanctioning of non-conforming behavior (see Sec. Methods 
for details) is not the only mechanism to support the evolution of norms. Other cooperation-enhancing 
mechanisms such as kin selection (based on genetic relationship) and group selection tend to transform a 
prisoner's dilemma into a harmony game (see Fig. [6]). Therefore, genetic relatedness and group selection 
are not ideal mechanisms to establish shared behavioral norms. They rather support the formation of 
subcultures. Moreover, the transformation of prisoner's dilemma interactions into a snowdrift game is 
expected to cause polarization or conflict (see Fig. [6]). 



PDs with punishment Stag Hunt + PD 




0.25 0.5 0.75 
P(t) 



0.250.50.75 
P(t) 



Figure 7. Illustration of different ways to establish a behavioral norm. (A) Prisoner's 
dilemma games with model parameters B = -1, C = -2, when population 1 is much stronger than 
population 2 (/ = 0.8) and individuals of both populations apply costly punishment, whenever the 
respective focal individual behaves cooperatively, but (from his or her perspective) the interaction 
partners does not [125] . The punishment cost was assumed to be /3 = 2.5, the punishment fine 7 = 5/3 
(see Sec. Methods for details). When starting with an initial condition in the blue or green areas, costly 
punishment can establish a behavioral norm, corresponding to the asymptotically stable points at (0,1) 
and (1,0). When starting in the red area, however, everybody in both populations will finally behave in 
a non-cooperative way, as indicated by the asymptotically stable point at (0,0). (B) Population 1 
playing a stag hunt game with model parameters B = -1 and C = 2, while individuals in population 2 
experience prisoner's dilemma interactions with B — -1 and C — -2, assuming equally strong 
populations (/ = 0.5). All individuals are expected to end up with behavior 1, which is preferred by 
population 1. However, as the flow line starting with the white circle and ending up in the 
asymptotically stable point (1,0) illustrates, the evolution of the behavioral norm can take a long time 
and unexpected detours. 



The evolution of language 119 is another example for the importance of norm-establishing social 



interactions, since successful communication requires norms, how words are used (the "evolution of 



meaning") 120 121 . In this connection, it is interesting to study whether the explosive development 
of language and culture in humans is due to their ability to transform interactions into norm-promoting 
stag hunt interactions. From this point of view, repeated interactions thanks to human settlements, the 
development of reputation mechanisms, and the invention of punishment institutions should have largely 



accelerated cultural evolution 122 ■ 124 



Another interesting research direction relates to the circumstance that people do not only follow 
norms — at the same time, they also create norms. This touches the issue of norm emergence (64|73p0p2& 
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128 . In order to address it, one also has to answer questions such as the following: How is the "content" of 



norms generated or selected, i.e. how does the prescription of a behavioral role or normative behavior come 
about [129| ? How and why do people start sanctioning non-conforming individuals, although this is costly, 
and why is there a tendency towards conformity at all 130 ? This is beyond the scope of this paper. The 
same applies to related questions such as the following: Do norms always establish a "Pareto" or a "system 
optimum" [56p3lHl33l ? Do norms always emerge, when they would be "functional" or beneficial [79|134 



Do they disappear, when they are not beneficial anymore? Do norms reduce or produce conflicts 31 



How can one explain local conformity, global diversity, and punctuated equilibria [51] ? These points will 
be addressed in a forthcoming publication, while the goal of this manuscript was to develop a unified 
theoretical concept allowing one to study the interaction of populations with incompatible interests. 
This became possible by analysis of multi-population game-dynamical (replicator) equations, which have 
been used here to address, besides the evolution of norms, the occurrence of polarization or conflict, 
the outbreak of revolutions and several other relevant questions, like the importance of the power of a 
population and of the previous history ("initial conditions") for the outcome of social interactions. 



Methods 

Multi-population game-dynamical replicator equations 



Multi-population game-dynamical replicator equations [38j|46^6l] describe the temporal evolution of the 
proportions (t) of individuals showing behavior i at time t in population a, assuming that more success- 
ful behaviors spread, as these are imitated by individuals of the same population at a rate proportional 
to the increase in the expected success [61 92 135 . The expected success is determined from the fre- 



quency of interactions between two behaviors i and j, and by the associated payoffs Afj . Focusing on 
the above-mentioned social dilemmas, in the case of two interacting populations a, J £ {1, 2} and two 
behavioral strategies i, j £ {1, 2}, we assume the following for interactions within the same population 
a: If two interacting individuals show the same behavior i, both will either receive the payoff r a or p a . If 
we have r a ^ p a , we call the behavior with the larger payoff r a "preferred" or "cooperative" , the other 
behavior "non-cooperative" or "uncooperative" . When one individual chooses the cooperative behavior 
and the interaction partner is uncooperative, the first one receives the payoff s a and the second one the 
payoff t a . To model conflicts of interests, we assume that population a = 1 prefers behavior i — 1 and 
population 2 prefers behavior 2. Therefore, if an individual of population 1 meets an individual belonging 
to population 2 and both show the same behavior i = 1, the first one will earn Ri and the second one 
P2, as behavior i = 1 is considered uncooperative in population 2. Analogously, for i = 2 they earn Pi 
and i?2- If the interaction partners choose different behaviors i and j, they earn S a , when the behavior 
corresponds to their cooperative behavior, otherwise they earn T a 1461. In mathematical notation, the 
payoff matrix (Ajj ) for individuals belonging to population 1 in interactions with other individuals of the 
same population is 



Interaction partner's behavior 

j = 1 (preferred) j = 2 

Focal i = 1 (preferred) r*i si 

agent's 

behavior i = 2 ti p\ 

while the payoff matrix (Ajj) for interactions with individuals of population 2 is 
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Interaction partner's behavior 
j = 1 .7=2 (preferred) 

(2) 

Focal i = 1 (preferred) i?i Si w 

agent's 

behavior i = 2 7\ Pi 

To reflect incompatible preferences of both populations, wc assume that the payoff matrix of individu- 
als belonging to population 2 is "inverted" or "mirrored" . When interacting with individuals of population 
1, the related payoff matrix (Afj) is 



Interaction partner's behavior 

j = 1 (preferred) j = 2 

Focal i = \ P 2 T 2 

agent's 

behavior i = 2 (preferred) S 2 R 2 

while the payoff matrix {Ajj) when interacting with individuals of population 2 is 

Interaction partner's behavior 

j = 1 .7 = 2 (preferred) 

Focal i = l p 2 t 2 

agent's 

behavior i = 2 (preferred) s 2 r 2 



(3) 



(4) 



Assuming constant preferences and fixed relative population strengths f a , the resulting coupled game- 
dynamical replicator equations for the temporal evolution of the proportion p{t) = p\{t) of cooperative 
individuals in population 1 and the fraction q{t) = q 2 of cooperative individuals in population 2 become 



dp(t) P(*)[l-P(*)] [6i/ + (ci-6i)/p(t) + C 1 (l-/) + (B 1 -C 1 )(l-/)g(t) 

saturation factors growth factor F(p,q) containing interaction effects 

and 

dq{t) q(t)[l - q(t)] [b 2 (l - f) + (c 2 - b 2 ){l - f)q{t) + C 2 f + (B 2 - C 2 )fp(t) 



(5) 



dt 



(6) 



saturation factors growth factor G(p, q) containing interaction effects 



Here, we have used the abbreviation f = f x = 1 — f 2 . As indicated above, this parameter can reflect 
the relative population size of population 1. More generally, however, Af-f h can be considered to represent 
"effective payoffs" , and / can be used to model the "power" of population 1 (which may not only depend 
on population size, but also on education, the availability of weapons or technologies, and other factors). 
b a = s a — Pa, B a = S a — P a , c a = r a — t a , and C a = R a — T a are payoff-dependent model parameters, 
which can be positive, negative, or zero. When setting b a = B a = B and c a = C a — C for simplicity, the 
payoff depends on the own behavior % and the behavior j of the interaction partner only, but not on the 
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population he/she belongs to (i.e. in- and out-group interactions just determine whether an interaction 
partner may be imitated or not, but they do not influence the payoff). Given the values of B and C used 
in our computer simulations, it is easily possible to construct related payoff matrices. Fixing values for 
P (e.g. P = 0) and for D = R - P, we have R(P, D) = P + D, T{P, D) = R - C = P + D - C, and 
S(P,D) = P + B. 

In reality, in-group and out-group interactions may, of course, affect the payoff as well. Such situations 
can be treated by choosing different values for the lower-case and upper-case parameters. It is obvious 
that this creates a multitude of additional cases, which deserve to be investigated in detail. However, 
before doing so, one first has to understand the basic case addressed in this study, which is already quite 
complicated. 



Specification of Costly Punishment and Effects of Group Pressure 



Let us now consider costly punishment analogously to the way it was specified in Ref. 125 . Then, we 
have si = S - Pi, t\ = T - 7l) Si = S - Pi - 72, S 2 = S - p 2 ~ 7i> *2 = T - 72, s 2 = S - P 2 . This 
specification assumes that someone who receives the low "sucker's payoff" S in the event of unilateral 
cooperation (from his/her point of view), will punish the respective interaction partner immediately, 
which modifies the payoffs. The punishment performed by an individual of population a reduces the 
payoff of his/her interaction partner by the fine 7 a > 0. However, punishment is costly (it needs some 
punishment effort), which reduces the punisher's payoff by f3 a > 0. Usually one assumes j a > (3 a . The 
correspondingly changed payoffs imply the parameters 6j = B — Pi , Ci = C + 71 , Bi = B — Pi — 72 , 
Ci = C and b 2 = B — p 2 , c 2 = C + 72, B 2 = B — /3 2 — 71, C 2 = C, which must be inserted into Eqs. ([5j 
and ([6]). Therefore, punishment transforms the prisoner's dilemma within a population into a stag hunt 
game, when c a = C + 7 a > 0, i.e. j a > \C\. The interaction with the other population remains a prisoner's 
dilemma, since B a < and C a < 0. Altogether, costly punishment results in the modified two-population 
game-dynamical equations 



dp{t) 
dt 



= p(t)[l - p(t)} F(p, q) - pif[l - p(t)} + 7 ifp(t) - (pi + 72 )(1 - f)q(t) 



(7) 



and 



dqjt) 
dt 



q(t)[l - q(t)} G(jp, q) - P 2 {1 - /)[1 - q(t)] + l2 (l - f)q(t) - (p 2 + 7l )/p(t) 



(8) 



These can generate dp/dt > and dq/dt > even for the prisoner's dilemma with B < and C 
< 0. While punishment in a one-population prisoner's dilemma can lead to cooperation 125 , in the 
mw/iz-population case it can cause normative behavior (see green and blue areas in Fig. [7j^). 

Rather than considering costly punishment as discussed before, one may also consider that individuals 
can apply group pressure to support conformity and discourage discoordinated behavior 19 70 . That 
could be reflected by subtracting a value 8 from the off-diagonal payoffs S and T or by adding S to 
the diagonal elements R and P. This results in the effective model parameters b a = B a = B — 5 and 
C + S 



C a 



117 . Therefore, if the group pressure S is large enough (namely, 6 > \ C\), a prisoner's 



dilemma with B < and C < is transformed into a stag hunt game with b a = B a < and c a = C a > 0. 



Discussion of Implicit Model Assumptions 

Any model has some underlying model assumptions, and even though they may not be exactly fulfilled, 
the resulting model can be a useful approximation. The evolutionary game theoretical model of this 
paper assumes that individuals show a certain behavior and stick to it until they change it due to 
social learning (e.g. imitation) or mutations (e.g. trial-and-error behavior). This appears applicable to 
situations of routine choice [124] and to situations, where individuals do not spend much time on analyzing 
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situations, but rather orient at what the others do. Crowd behavior and certain kinds of public opinion 



formation seem to be good examples for this 61 . The approach should also be applicable to many kinds 
of culturally acquired behaviors, including a considerable number of behavioral conventions and norms. 

In contrast to evolutionary game theory, classical game theory assumes complex, strategic decision- 
making processes based on utility maximization. These decision-making processes usually consider indi- 
vidual preferences of interaction partners and other aspects. In spite of this difference, both approaches 
lead to mutually consistent outcomes in many cases. For this reason, it should not matter for our main 
conclusions whether the analysis is based on classical or evolutionary game theory. Indeed both kinds of 
game-theoretical analysis are expected to show the four archetypical types of system behaviors identified 
in our paper. In this connection, it is interesting to note the following j2j: 

1. Every Nash equilibrium is a fixed point of the replicator dynamics and the game-dynamical equation. 
(At a Nash equilibrium, which may also correspond to a mixed strategy, no player can improve the 
payoff by changing the strategy unilaterally.) 

2. Every (asymptotically or neutrally) stable fix point of the replicator equation is a Nash equilibrium. 

3. The fix points (and their stability properties) imply the main conclusions of our paper, as they 
determine the features of the system dynamics, which are reflected by the basins of attraction and 
the flow lines. 

Further assumptions underlying the multi-population evolutionary game-theoretical model become 
obvious in the mathematical derivation of the equations. One- or multi-population replicator equations 



may describe the spreading of more successful individuals via a higher reproduction rate (see, e.g. 35 



However, they may also reflect social learning, namely by the imitation of more successful behaviors 



46 61 92 . The above model equations result in case of proportional imitation 61 135 . They assume 
that interactions take place in large, well-mixed populations, usually between different individuals. If 
effects of repeated interactions (and, thereby, a "shadow of the future") shall be taken into account, this 



can be done by modifying the payoffs accordingly [3| |117 . A similar thing applies to reputation effects 



network reciprocity, group selection, etc. (see also Fig. K3J). It is furthermore possible to generalize the 



example, 14.137 138 



approach to finite populations 136 and to populations with spatial or network interactions (see, for 



Entities belonging to different populations differ in two aspects: They earn different payoffs, and they 
imitate different entities (namely, better-performing entities belonging to the own population, assuming 
that it would not necessarily be wise to copy behaviors of individuals with different preferences and payoff 
functions). If, additionally, in-group interactions shall be distinguished from out-group interactions (in 
the sense that not only the actual behavior, but also the preferred behavior of the interaction partner 
matters), one has to specify the parameters b a and c a differently from the parameters B a and C a . In this 
way, one can even consider cases, where both populations play different games (see Fig. up). 
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